Method and apparatus for video stabilization by compensating for view direction of camera

ABSTRACT

A video stabilization method includes: measuring an inter-frame camera motion based on a difference angle of a relative camera view direction in comparison with a reference camera view direction in each frame of a frame sequence of a video; generating a camera motion path of the frame sequence by using the inter-frame camera motion and determining a camera view direction adjustment angle based on a user&#39;s view direction by using the camera motion path; and compensating for the camera view direction by using the camera view direction adjustment angle in each frame.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/412,938, filed on Nov. 12, 2010, in the United StatesPatent and Trademark Office, and priority from Korean Patent ApplicationNo. 10-2011-0006488, filed on Jan. 21, 2011, in the Korean IntellectualProperty Office, the disclosures of which are incorporated herein byreference in their entireties.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Apparatuses and methods consistent with exemplary embodiments accordingto the present inventive concept relate to video stabilizationtechnology for compensating for movement and shaking of a camera.

2. Description of the Related Art

Conventional video stabilization technology is classified into a videostabilization scheme based on 2-dimensional (2D) analysis and a schemebased on 3-dimensional (3D) analysis.

The video stabilization scheme based on 2D analysis includes estimatinga motion, setting a camera motion path through the estimated motion, andgenerating an image according to the camera motion path. In theestimating of the motion, the motion is measured by analyzing a capturedimage with a 2D motion model. Although an image is actually obtained byprojecting a 3D space into a 2D image in a camera, a 3D motion cannot becorrectly modeled with the 2D motion model. Thus, although the videostabilization scheme based on 2D analysis cannot perfectly perform videostabilization, the video stabilization scheme based on 2D analysis canderive an effective result with a relatively small amount ofcomputation.

The video stabilization scheme based on 3D analysis generallyreconstructs 3D space information and camera position information from a2D image by using motion structure analysis technology. A new cameraposition is defined by using the reconstructed information, and imageinformation regarding the new camera position is synthesized. Althoughthe video stabilization scheme based on 3D analysis requires aconsiderably greater amount of computation than the video stabilizationscheme based on 2D analysis, the video stabilization scheme based on 3Danalysis can derive correct video stabilization. However, an image uponwhich video stabilization has been performed based on 3D analysis mayhave a problem of image distortion in a partial area of the image.

SUMMARY OF THE INVENTION

Exemplary embodiments according to the present inventive concept providea video stabilization method and apparatus for providing a screensimilar to an actual motion perceived by human vision while correctlyand quickly compensating for movement and shaking of a camera.

According to an aspect of one or more exemplary embodiments, there isprovided a video stabilization method including: measuring aninter-frame camera motion based on a difference angle of a relativecamera view direction in comparison with a reference camera viewdirection in each frame of a frame sequence of a video; generating acamera motion path of the frame sequence by using the inter-frame cameramotion and determining a camera parameter for each frame by using thecamera motion path; and compensating for a camera motion by using thecamera parameter for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: measuring a cameramotion in each frame of a frame sequence of a video; generating a cameramotion path of the frame sequence by using the camera motion in eachframe and determining a camera view direction adjustment angle between auser's view direction and a camera view direction by using the cameramotion path; and compensating for the camera view direction by using thecamera view direction adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: measuring a cameramotion in each frame of a frame sequence of a video; determining atarget area of a view direction in the frame sequence; generating acamera motion path of the frame sequence by using the camera motion ineach frame and determining a camera view direction adjustment anglebased on the view direction oriented to the target area by using thecamera motion path; and compensating for a camera view direction byusing the camera view direction adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: measuring aninter-frame camera motion based on a difference angle of a relativecamera view direction in comparison with a reference camera viewdirection in each frame of a frame sequence of a video; generating acamera motion path of the frame sequence by using the inter-frame cameramotion and determining a camera view direction adjustment angle based ona user's view direction by using the camera motion path; andcompensating for the camera view direction by using the camera viewdirection adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: receiving aselection request of a target area of a view direction in a framesequence from a user through a user interface; measuring an inter-framecamera motion based on a difference angle of a relative camera viewdirection in comparison with a reference camera view direction in eachframe of a frame sequence of a video; generating a camera motion path ofthe frame sequence by using the inter-frame camera motion anddetermining a camera view direction adjustment angle based on a viewdirection oriented to the target area by using the camera motion path;and compensating for the camera view direction by using the camera viewdirection adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: receiving aselection request of at least one of a margin area size and astabilization level for video stabilization from a user through a userinterface; and stabilizing a video of a camera by measuring a relativeinter-frame camera motion based on a reference camera view direction ina frame sequence of the video based on the at least one of the marginarea size and the stabilization level, determining a camera viewdirection adjustment angle based on a view direction oriented to atarget area, and compensating for a camera view direction by using thecamera view direction adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization method including: stabilizing a videoof a camera by measuring a relative inter-frame camera motion based on areference camera view direction in a frame sequence of the video,determining a camera view direction adjustment angle based on a viewdirection oriented to a target area, and compensating for a camera viewdirection by using the camera view direction adjustment angle for eachframe; measuring a stabilization related index while performing thestabilization of the video of the camera; and warning in real-time ifthe stabilization related index exceeds a predetermined threshold.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a camera motionmeasurer which measures an inter-frame camera motion based on adifference angle of a relative camera view direction in comparison witha reference camera view direction in each frame of a frame sequence of avideo; a camera parameter determiner which generates a camera motionpath of the frame sequence by using the inter-frame camera motion anddetermines a camera parameter for each frame by using the camera motionpath; and a camera motion compensator which compensates for a cameramotion by using the camera parameter for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a camera motionmeasurer which measures a camera motion in each frame of a framesequence of a video; a camera view direction adjustment angle determinerwhich generates a camera motion path of the frame sequence by using thecamera motion in each frame and determines a camera view directionadjustment angle based on a user's view direction by using the cameramotion path; and a camera view direction compensator which compensatesfor a camera view direction by using the camera view directionadjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a camera motionmeasurer which measures a camera motion in each frame of a framesequence of a video; a view direction target area determiner whichdetermines a target area of a view direction in the frame sequence; acamera view direction adjustment angle determiner which generates acamera motion path of the frame sequence by using the camera motion ineach frame and determines a camera view direction adjustment angle basedon the view direction oriented to the target area by using the cameramotion path; and a camera view direction adjustment angle compensatorwhich compensates for a camera view direction by using the camera viewdirection adjustment angle based on the view direction oriented to thetarget area for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a camera motionmeasurer which measures an inter-frame camera motion by determining adifference angle of a relative camera view direction in comparison witha reference camera view direction in each frame of a frame sequence of avideo; a camera view direction adjustment angle determiner whichgenerates a camera motion path of the frame sequence by using theinter-frame camera motion and determines a camera view directionadjustment angle based on a user's view direction by using the cameramotion path; and a camera view direction compensator which compensatesfor the camera view direction by using the camera view directionadjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a view directiontarget area user input unit which receives a selection request of atarget area of a view direction in a frame sequence from a user througha user interface; a camera motion measurer which measures an inter-framecamera motion based on a difference angle of a relative camera viewdirection in comparison with a reference camera view direction in eachframe of a frame sequence of a video; a camera view direction adjustmentangle determiner which generates a camera motion path of the framesequence by using the inter-frame camera motion and determines a cameraview direction adjustment angle based on a view direction oriented tothe target area by using the camera motion path; and a camera viewdirection compensator which compensates for the camera view direction byusing the camera view direction adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a margin areauser input unit which receives a selection request of at least one of amargin area size and a stabilization level for video stabilization froma user through a user interface; and a video camera stabilizer whichstabilizes a video of a camera by measuring a relative inter-framecamera motion based on a reference camera view direction in a framesequence of a video based on the at least one of the margin area sizeand the stabilization level, determines a camera view directionadjustment angle based on a view direction oriented to a target area,and compensates for a camera view direction by using the camera viewdirection adjustment angle for each frame.

According to another aspect of one or more exemplary embodiments, thereis provided a video stabilization apparatus including: a video camerastabilizer which stabilizes a video of a camera by measuring a relativeinter-frame camera motion based on a reference camera view direction ina frame sequence of the video, determines a camera view directionadjustment angle based on a view direction oriented to a target area,and compensates for a camera view direction by using the camera viewdirection adjustment angle for each frame; a stabilization related indexmeasurer which measures a stabilization related index while performingthe stabilization of the video of the camera; and a video stabilizationwarning unit which warns in real-time if the stabilization related indexexceeds a predetermined threshold.

According to another aspect of one or more exemplary embodiments, thereis provided a computer-readable recording medium storing acomputer-readable program for executing the video stabilization method.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventiveconcept will become more apparent by describing in detail exemplaryembodiments thereof with reference to the attached drawings in which:

FIG. 1 is a block diagram of a video stabilization apparatus accordingto an exemplary embodiment of the present inventive concept;

FIG. 2 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 3 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 4 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 5 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 6 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 7 is a block diagram of a video stabilization apparatus accordingto another exemplary embodiment of the present inventive concept;

FIG. 8 is a diagram for describing a camera motion measuring methodaccording to an exemplary embodiment of the present inventive concept;

FIG. 9 is a diagram for describing a method of measuring a camerarotation angle in a Z-axis direction, according to an exemplaryembodiment of the present inventive concept;

FIG. 10 is a diagram showing a camera motion path, and FIG. 11 is adiagram for describing a conventional video stabilization method;

FIG. 12 is a diagram showing a camera motion path obtained by adjustinga camera motion based on a global camera view direction of the cameramotion path, according to an exemplary embodiment of the presentinventive concept;

FIG. 13 is a diagram showing a camera motion path obtained by adjustinga camera motion based on a camera view direction oriented to a targetarea, according to an exemplary embodiment of the present inventiveconcept;

FIG. 14 is a diagram showing user interfaces for adjusting a size of amargin area and a degree of video stabilization according to a videostabilization operation, according to an exemplary embodiment of thepresent inventive concept;

FIG. 15 is a diagram showing a user interface for a user to set a targetarea of a camera view direction for a video stabilization operation,according to an exemplary embodiment of the present inventive concept;

FIG. 16 is a diagram showing a user interface for providing a warningrelating to an excessive video stabilization operation while performinga video stabilization operation, according to an exemplary embodiment ofthe present inventive concept;

FIG. 17 is a flowchart of a video stabilization method according to anexemplary embodiment of the present inventive concept;

FIG. 18 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept;

FIG. 19 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept;

FIG. 20 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept;

FIG. 21 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept;

FIG. 22 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept; and

FIG. 23 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, video stabilization apparatuses according to variousexemplary embodiments of the present inventive concept are describedwith reference to FIGS. 1 to 7. Detailed operations of the videostabilization apparatuses according to various exemplary embodiments ofthe present inventive concept are described with reference to FIGS. 8 to13. User interfaces for the video stabilization apparatuses according tovarious exemplary embodiments of the present inventive concept areillustrated with reference to FIGS. 14 to 16. Video stabilizationmethods according to various exemplary embodiments of the presentinventive concept are described with reference to FIGS. 17 to 23.

Video stabilization apparatuses 100 and 400 according to exemplaryembodiments illustrated in FIGS. 1 and 4 provide a video stabilizationscheme for compensating for a camera motion by using only a relativeinter-frame camera rotation angle measured as an inter-frame cameramotion.

Video stabilization apparatuses 200, 300, and 400 according to exemplaryembodiments illustrated in FIGS. 2, 3, and 4, respectively, provide avideo stabilization scheme for compensating for shaking of a camera byusing only a difference between a camera view direction of a currentframe and a user's predicted view direction on a camera motion path.

Video stabilization apparatuses 500, 600, and 700 according to exemplaryembodiments illustrated in FIGS. 5, 6, and 7, respectively, include anyone of the video stabilization apparatuses 100 to 400 according to theembodiments of FIGS. 1 to 4 and provide a video stabilization schemeincluding a user interface.

FIG. 1 is a block diagram of the video stabilization apparatus 100according to an exemplary embodiment of the present inventive concept.

Referring to FIG. 1, the video stabilization apparatus 100 may includean inter-frame camera motion measurer 110, a camera parameter determiner120, and a camera motion compensator 130. The video stabilizationapparatus 100 may output a stabilized video by compensating for a cameramotion by using only a relative motion between camera view directions oftwo frames as camera motion information.

The inter-frame camera motion measurer 110 may acquire a frame sequenceof a video and measure an inter-frame camera motion in each frame of theframe sequence. The camera parameter determiner 120 may generate acamera motion path of the frame sequence by using the inter-frame cameramotion measured by the inter-frame camera motion measurer 110 anddetermine a camera parameter for each frame according to the cameramotion path. The camera motion compensator 130 may compensate for thecamera motion by using the camera parameter determined by the cameraparameter determiner 120 for each frame.

The inter-frame camera motion measurer 110 may determine relative cameraview direction information based on a reference camera view direction asinter-frame camera motion information in each frame. For example, theinter-frame camera motion measurer 110 may determine a difference anglebetween a camera view direction of a current frame and the referencecamera view direction as the inter-frame camera motion information.

The reference camera view direction for determining a relative cameramotion in a camera view direction of a current frame may be a cameraview direction of a previous frame. For example, the inter-frame cameramotion measurer 110 may determine a relative camera view directiondifference angle between a previous frame and a current frame in everypair of temporally successive frames in a frame sequence.

Alternatively, the reference camera view direction may be commonlydetermined for the entire frame sequence. For example, the inter-framecamera motion measurer 110 may measure an average of camera viewdirection difference angles between every pair of successive frames inthe frame sequence and determine the average difference angle of cameraview directions as the reference camera view direction. Thus, theinter-frame camera motion measurer 110 may measure a relative cameraview direction, i.e., a difference angle A-B between an averagedifference angle A of camera view directions and a camera view directionB of a current frame, as camera motion information in each frame basedon the average difference angle of the camera view directions of theframe sequence.

The inter-frame camera motion measurer 110 may determine a camera viewdirection difference angle as the camera motion information withrotation angles in each of the X, Y, and Z directions. The inter-framecamera motion measurer 110 may determine X-axis and Y-axis components ofa camera motion by measuring relative angle changes in the X-axisdirection and the Y-axis direction between a camera view direction of acurrent frame and the reference camera view direction.

The inter-frame camera motion measurer 110 may determine a camerarotation angle in the Z-axis direction by using a camera motion in astraight-lined direction occurring in predetermined areas on a Z-axisplane by a camera rotation in the Z-axis direction. At least one samplearea for analyzing the camera motion in the straight-lined directionoccurring by the camera rotation in the Z-axis direction may beselected.

The inter-frame camera motion measurer 110 may determine a total errorper camera rotation angle by determining errors occurring between sampleareas of a reference frame and a current frame in the reference cameraview direction for each camera rotation angle within a predeterminedrange among camera rotation angles in the Z-axis direction and summingthe errors of the sample areas. The inter-frame camera motion measurer110 may detect a camera rotation angle to minimize a total error ofsample areas from among camera rotation angles in the Z-axis directionwithin a predetermined range and determine the detected camera rotationangle as a relative camera rotation angle in the Z-axis direction.

The inter-frame camera motion measurer 110 may determine a relativeinter-frame camera view direction difference angle by using frames,i.e., images, of the frame sequence of the video. The inter-frame cameramotion measurer 110 according to another exemplary embodiment of thepresent inventive concept may predict a camera motion by using a sensor.For example, the inter-frame camera motion measurer 110 may determine arelative inter-frame camera view direction difference angle by usingcamera motion information acquired by using a gyro sensor, a G-sensor,or an accelerometer.

The camera parameter determiner 120 may generate a camera motion path ofthe frame sequence by accumulating the inter-frame camera motioninformation measured by the inter-frame camera motion measurer 110. Thecamera parameter determiner 120 may also generate a global camera motionpath of a camera motion path. For example, the global camera motion pathmay be generated through low-pass filtering of the camera motion path. Adegree of stabilization may be adjusted according to a window size andstrength for the low-pass filtering.

Since the inter-frame camera motion measured by the inter-frame cameramotion measurer 110 is measured by using a relative camera motion suchas a difference angle of camera view directions, a camera view directionand a global camera view direction on the camera motion path generatedby the camera parameter determiner 120 may also be defined by a relativecamera motion, e.g., a difference angle of a relative camera viewdirection in comparison with a predetermined reference camera viewdirection.

The camera parameter determiner 120 may determine a camera adjustmentvalue based on the global camera motion path in each frame. For example,when the camera motion path is changed to the global camera motion paththrough video stabilization, the camera parameter determiner 120 maydetermine a difference between an actual camera motion path generated byaccumulating inter-frame camera motions and the global camera motionpath as the camera adjustment value.

The camera motion path may be defined by a camera view vector, i.e., acamera position and a camera view direction, on the camera motion path.Thus, the camera parameter determiner 120 may determine a cameraposition displacement between the actual camera motion path and theglobal camera motion path and a relative direction difference betweencamera view directions thereof as the camera adjustment value.

The camera parameter determiner 120 may determine a relative differenceangle, i.e., a rotation angle, between the actual camera motion path andthe global camera motion path by using information regarding thedifference angle between a current camera view direction on the cameramotion path and the reference camera view direction that is receivedfrom the inter-frame camera motion measurer 110. For example, the cameraparameter determiner 120 may determine a camera view directionadjustment angle by summing the difference angle between the currentcamera view direction and the reference camera view direction and arotation angle between the global camera view direction on the cameramotion path and the reference camera view direction.

The camera adjustment value determined for each frame by the cameraparameter determiner 120 may be outputted to the camera motioncompensator 130 as a camera parameter in each frame. In particular, theinter-frame camera motion measurer 110 may extract only a variation,i.e., a rotation angle, of a camera view direction as a camera parameterand transmit the extracted rotation angle of the camera view directionto the camera motion compensator 130.

The camera motion compensator 130 may compensate for a camera motion ineach frame by using the camera parameter in each frame. For example, thecamera motion compensator 130 may rotate the camera view direction onthe actual camera motion path by the camera view direction adjustmentangle in each frame by using the camera parameter in each frame.

The video stabilization apparatus 100 may measure relative inter-framecamera motions in a frame sequence of a single point of view video,generate a camera motion path by accumulating the relative inter-framecamera motions, and perform video stabilization on the camera motionpath. Here, the camera motion path may be predicted by measuring arelative camera motion between temporally successive frames in the framesequence of the single point of view video.

In addition, the video stabilization apparatus 100 may apply a videostabilization scheme to a stereoscopic video. In detail, the inter-framecamera motion measurer 110 of the video stabilization apparatus 100 maymeasure an inter-viewpoint camera motion for left-viewpoint frames andright-viewpoint frames corresponding to each other in the stereoscopicvideo. A relative direction difference, i.e., a difference angle, of theremaining viewpoint camera view direction based on a reference viewpointcamera view direction from among a left-viewpoint and a right-viewpointmay be determined. Thus, the video stabilization apparatus 100 mayperform a video stabilization operation of the stereoscopic video by theinter-frame camera motion measurer 110 measuring an inter-viewpointcamera motion in each sequential frame.

FIG. 2 is a block diagram of the video stabilization apparatus 200according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 2, the video stabilization apparatus 200 may include acamera motion measurer 210, a camera view direction adjustment angledeterminer 220, and a camera view direction compensator 230. The videostabilization apparatus 200 may output a stabilized video by generatinga camera motion path by using a camera motion and compensating for thecamera motion by adjusting only a camera view direction excluding acamera position on the camera motion path.

The camera motion measurer 210 may receive a frame sequence of a videoand measure a camera motion in each frame of the frame sequence. Thecamera view direction adjustment angle determiner 220 may generate acamera motion path of the frame sequence by using the camera motion ineach frame that is measured by the camera motion measurer 210, anddetermine a camera view direction adjustment angle based on a user'sview direction by using the camera motion path. The camera viewdirection compensator 230 may compensate for a camera view direction ineach frame by using the camera view direction adjustment angledetermined by the camera view direction adjustment angle determiner 220.Accordingly, the video stabilization apparatus 200 may complete a videostabilization operation.

The camera motion measurer 210 may determine a camera view vector ineach frame to measure the camera motion in each frame. The camera viewdirection adjustment angle determiner 220 may generate the camera motionpath of the frame sequence by accumulating the camera motion in eachframe that is determined by the camera motion measurer 210. Thus, thecamera motion path generated by the camera view direction adjustmentangle determiner 220 may include information regarding the camera viewvector in each frame.

The camera view direction adjustment angle determiner 220 may predictthe user's view direction by using the camera motion path of the framesequence. For example, the camera view direction adjustment angledeterminer 220 may separate a global camera view direction and a localcamera view direction from the generated camera motion path anddetermine the global camera view direction as the user's view direction.

There are various methods of separating the global camera view directionfrom the camera motion path of the frame sequence. For example, theglobal camera view direction of the frame sequence may be determinedthrough low-pass filtering of the camera motion path, and a camera viewdirection on a global camera motion path may be determined as the globalcamera view direction. However, a method of determining the globalcamera motion path is not limited to the low-pass filtering.

The camera view direction adjustment angle determiner 220 may determinea camera parameter by using only the camera view direction excluding acamera position on the camera motion path. That is, the camera viewdirection adjustment angle determiner 220 may determine a differenceangle between the user's view direction and a current camera viewdirection on the camera motion path as the camera view directionadjustment angle.

The camera view direction on the camera motion path may be adjusted tobe oriented toward a frame center. That is, the camera view directionadjustment angle determiner 220 may detect the camera view directionadjustment angle to move a current camera view direction to the framecenter.

The camera view direction compensator 230 may compensate for a cameramotion by rotating a current camera view direction of a frame by thecamera view direction adjustment angle determined by the camera viewdirection adjustment angle determiner 220.

Since the video stabilization apparatus 200 adjusts only the camera viewdirection excluding the camera position on the camera motion path, videostabilization may be performed with an effect that only a camera viewdirection on an actual camera motion path is moved to the user's viewdirection while a camera position on the actual camera motion path ismaintained.

In addition, the video stabilization apparatus 200 may also perform avideo stabilization operation of a stereoscopic video by the cameramotion measurer 210 measuring an inter-viewpoint camera motion in eachsequential frame for left-viewpoint frames and right-viewpoint framescorresponding to each other in the stereoscopic video.

FIG. 3 is a block diagram of the video stabilization apparatus 300according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 3, the video stabilization apparatus 300 may include acamera motion measurer 310, a view direction target area determiner 315,a camera view direction adjustment angle determiner 320, and a cameraview direction compensator 330. The video stabilization apparatus 300may compensate for a camera motion by adjusting only a camera viewdirection on a camera motion path, like the video stabilizationapparatus 200, and may further arbitrarily select a target area of thecamera view direction.

The camera motion measurer 310 may receive a frame sequence of a videoand measure a camera motion in each frame of a frame sequence. The viewdirection target area determiner 315 may determine a target area towhich a camera view direction is oriented in the frame sequence. Thecamera view direction adjustment angle determiner 320 may generate acamera motion path of the frame sequence by using the camera motion ineach frame that is measured by the camera motion measurer 310, anddetermine a camera view direction adjustment angle based on a cameraview direction oriented to the target area of each point on the cameramotion path. The camera view direction compensator 330 may compensatefor a camera view direction in each frame by using the camera viewdirection adjustment angle determined by the camera view directionadjustment angle determiner 320.

The camera motion measurer 310 may determine a camera view vector ineach frame to measure the camera motion in each frame.

The view direction target area determiner 315 may determine the targetarea to which the camera view direction is oriented by analyzing acamera view direction in each frame in the frame sequence. For example,the view direction target area determiner 315 may detect arepresentative target area to which camera view directions are orientedor a target area of a global camera view direction by analyzing thecamera view directions in the frame sequence.

Alternatively, the view direction target area determiner 315 mayarbitrarily select the target area of the camera view direction ordetermine the target area of the camera view direction based on arequest for selecting the target area of the camera view direction,which is received from the outside. The view direction target areadeterminer 315 may transmit information regarding the target area of thecamera view direction to the camera view direction adjustment angledeterminer 320.

The camera view direction adjustment angle determiner 320 may generatethe camera motion path of the frame sequence by using the camera motionin each frame that is determined by the camera motion measurer 310, anddetermine the camera view direction adjustment angle based on the cameraview direction oriented to the target area by using the camera motionpath.

The camera view direction adjustment angle determiner 320 may generatethe camera motion path of the frame sequence by accumulating the cameramotion in each frame that is determined by the camera motion measurer310. The camera motion path generated by the camera view directionadjustment angle determiner 320 may include information regarding thecamera view vector in each frame.

The camera view direction adjustment angle determiner 320 may predict auser's view direction by using the camera motion path. In addition, whenthe camera view direction adjustment angle determiner 320 receivesinformation regarding the target area of the camera view direction fromthe view direction target area determiner 315, the camera view directionadjustment angle determiner 320 may determine a camera view directionadjustment angle based on the target area of the camera view directionin each frame.

The camera view direction adjustment angle determiner 320 may determinethe camera view direction adjustment angle based on the target area ofthe camera view direction by using the user's view direction or theglobal camera view direction.

The camera view direction adjustment angle determiner 320 may separatethe global camera view direction and a local camera view direction froma current camera view direction of the camera motion path. The cameraview direction adjustment angle determiner 320 may determine a firstadjustment angle of the camera view direction based on the global cameraview direction in each frame.

The camera view direction adjustment angle determiner 320 may analyze adisplacement or a coordinate difference between a point to which theglobal camera view direction is oriented and the target area in a framebased on the information regarding the target area of the camera viewdirection that is received from the view direction target areadeterminer 315. The camera view direction adjustment angle determiner320 may detect a difference angle between the global camera viewdirection and the camera view direction oriented to the target area byusing the displacement or the coordinate difference between the point towhich the global camera view direction is oriented and the target areaand determine the detected difference angle as a second adjustmentangle.

The camera view direction adjustment angle determiner 320 may determinethe camera view direction adjustment angle by summing the firstadjustment angle and the second adjustment angle of the camera viewdirection in each frame. The camera view direction compensator 330 maycompensate for camera shaking by compensating for the camera viewdirection based on information regarding the camera view directionadjustment angle output from the camera view direction adjustment angledeterminer 320.

In addition, the video stabilization apparatus 300 may also perform avideo stabilization operation of a stereoscopic video by the cameramotion measurer 310 measuring an inter-viewpoint camera motion in eachsequential frame for left-viewpoint frames and right-viewpoint framescorresponding to each other in the stereoscopic video.

FIG. 4 is a block diagram of the video stabilization apparatus 400according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 4, the video stabilization apparatus 400 may includean inter-frame camera motion measurer 410, a camera view directionadjustment angle determiner 420, and a camera view direction compensator430. The video stabilization apparatus 400 may output a stabilized videoby generating a camera motion path through measurement of a cameramotion with a relative motion between camera view directions of twoframes and compensating for the camera motion through adjustment of onlya camera view direction excluding a camera position on the camera motionpath.

The inter-frame camera motion measurer 410 may receive a frame sequenceof a video and measure an inter-frame camera motion based on adifference angle of a relative camera view direction in comparison witha reference camera view direction in each frame. The camera viewdirection adjustment angle determiner 420 may generate a camera motionpath of the frame sequence by using information regarding theinter-frame camera motion that is acquired by the inter-frame cameramotion measurer 410, and determine a camera view direction adjustmentangle based on a target camera view direction by using the camera motionpath. The camera view direction compensator 430 may compensate for acamera view direction in each frame by using the camera view directionadjustment angle determined by the camera view direction adjustmentangle determiner 420.

The inter-frame camera motion measurer 410 may perform the sameoperation as the inter-frame camera motion measurer 110. That is, theinter-frame camera motion measurer 410 may determine a difference angleof relative camera view directions between every pair of temporallysuccessive frames in the frame sequence as inter-frame camera motioninformation. The inter-frame camera motion measurer 410 may transmit theinter-frame camera motion information to the camera view directionadjustment angle determiner 420.

The inter-frame camera motion measurer 410 may determine the cameramotion information by measuring relative camera view directions based onthe reference camera view direction in each of the X, Y, and Zdirections. The inter-frame camera motion measurer 410 may determine arelative inter-frame camera view direction difference angle by usingframes, i.e., images, of the frame sequence of the video. Alternatively,the inter-frame camera motion measurer 410 may determine the relativeinter-frame camera view direction difference angle by using a cameramotion predicted using a sensor.

The camera view direction adjustment angle determiner 420 may generatethe camera motion path of the frame sequence by accumulating inter-framecamera motions based on the inter-frame camera motion informationreceived from the inter-frame camera motion measurer 410. The cameramotion path generated by the camera view direction adjustment angledeterminer 420 may be defined by only relative camera view directiondifference values excluding camera position information.

Current camera view direction information may be extracted from thecamera motion path. The camera view direction adjustment angledeterminer 420 may determine a camera view direction adjustment angle tochange a current camera view direction to a desired camera viewdirection.

The camera view direction adjustment angle determiner 420 may predict auser's view direction from a current camera view direction on the cameramotion path. The camera view direction adjustment angle determiner 420may determine a global camera motion path by using the camera motionpath and determine a camera view direction on the global camera motionpath as the user's view direction. The camera view direction adjustmentangle determiner 420 may determine a first adjustment angle of thecamera view direction based on the user's view direction (i.e., theglobal camera view direction) in each frame.

The video stabilization apparatus 400 may further include a viewdirection target area determiner 415. When the camera view directionadjustment angle determiner 420 receives target area information fromthe view direction target area determiner 415, the camera view directionadjustment angle determiner 420 may determine a camera view directionadjustment angle based on a camera view direction oriented to a targetarea of each point on the camera motion path of the frame sequence.

In this case, the camera view direction adjustment angle determiner 420may determine the camera view direction adjustment angle by summing thefirst adjustment angle of the camera view direction based on the user'sview direction (i.e., the global camera view direction) and a secondadjustment angle of the camera view direction based on the camera viewdirection oriented to the target area, in a similar method to thatemployed by the camera view direction adjustment angle determiner 320 ofFIG. 3.

In addition, the camera view direction adjustment angle determiner 420may determine the camera view direction adjustment angle by usingrotation angle information between a current camera view direction onthe camera motion path and the reference camera view direction, which isreceived from the inter-frame camera motion measurer 410.

For example, the camera view direction adjustment angle determiner 420may determine the camera view direction adjustment angle by summing arotation angle between a current camera view direction on the cameramotion path and the reference camera view direction and a rotation anglebetween the reference camera view direction and the camera viewdirection oriented to the target area. Since the current camera viewdirection is defined by a relative rotation angle based on the referencecamera view direction, the camera view direction adjustment angle may beeasily determined by using a relative rotation angle between thereference camera view direction and the camera view direction orientedto the target area.

The camera view direction compensator 430 may compensate for a cameramotion by rotating a camera view direction by a camera view directionadjustment angle in each frame.

The view direction target area determiner 415 may set the global cameraview direction on the camera motion path to an initial value of thecamera view direction oriented to the target area.

The video stabilization apparatus 400 may also perform a videostabilization operation of a stereoscopic video by the camera motionmeasurer 410 measuring an inter-viewpoint camera motion in eachsequential frame for left-viewpoint frames and right-viewpoint framescorresponding to each other in the stereoscopic video, in a similarmethod to those employed by the video stabilization apparatuses 100,200, and 300 as described above.

FIG. 5 is a block diagram of the video stabilization apparatus 500according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 5, the video stabilization apparatus 500 may include aview direction target area user input unit 520 and a video stabilizationdevice 510. The video stabilization device 510 may be the same as thevideo stabilization apparatus 300 or 400 described above.

The view direction target area user input unit 520 may receive aselection request of a target area of a camera view direction in a framesequence from a user through a user interface. The video stabilizationdevice 510 may receive information regarding the target area of thecamera view direction that is transmitted from the view direction targetarea user input unit 520. The video stabilization device 510 may adjusta camera view direction based on the camera view direction oriented tothe target area in the same method as that employed by the videostabilization apparatus 300 or 400. Accordingly, the video stabilizationapparatus 500 may perform video stabilization for generating a resultsimilar to vision shaking perceived by a human being by adjusting acamera view direction of a video to be oriented to a target area desiredby a user.

FIG. 6 is a block diagram of the video stabilization apparatus 600according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 6, the video stabilization apparatus 600 may include amargin area user input unit 620 and a video stabilization device 610.The video stabilization device 610 may be the same as the videostabilization apparatus 100, 200, 300, or 400 described above.

The margin area user input unit 620 may receive a selection request ofat least one of a margin area size and a stabilization level for videostabilization from a user through a user interface. The margin area userinput unit 620 may determine the stabilization level based on the marginarea size selected by the user or determine the margin area size basedon the stabilization level selected by the user. Alternatively, themargin area user input unit 620 may receive a selection request of apredetermined set of a margin area size and a stabilization level fromthe user.

The video stabilization device 610 may receive information regarding atarget area of a camera view direction that is transmitted from themargin area user input unit 620. The video stabilization device 610 maycompensate for a camera motion by adjusting a camera view directionbased on a predetermined camera view direction in the same method asthat employed by the video stabilization apparatus 100, 200, 300, or400. However, the video stabilization device 610 may limit a camera viewdirection adjustment amount based on the at least one of the margin areasize and the stabilization level. For example, a measured magnitude ofthe camera motion, a measured magnitude of a camera view directionadjustment angle, and a variation due to the compensation for the cameraview direction may be limited based on the margin area size or thestabilization level.

Accordingly, the video stabilization apparatus 600 may prevent excessivevideo stabilization while performing video stabilization similar tovision shaking perceived by a human being by adjusting a camera viewdirection of a video.

FIG. 7 is a block diagram of the video stabilization apparatus 700according to another exemplary embodiment of the present inventiveconcept.

Referring to FIG. 7, the video stabilization apparatus 700 may include avideo stabilization device 710, a stabilization index measurer 720, anda video stabilization warning unit 730.

The video stabilization device 710 may be the same as the videostabilization apparatus 100, 200, 300, 400, 500 or 600 described above.That is, the video stabilization device 710 may stabilize a cameramotion of a video by measuring a camera motion in each frame of thevideo, generating a camera motion path by using the camera motions,determining a camera view direction adjustment angle based on a desiredcamera view direction in each frame by using the camera motion path, andcompensating for a camera view direction in each frame.

The stabilization index measurer 720 may measure a stabilization relatedindex by sensing a video camera stabilization operation of the videostabilization device 710 in real-time. The video stabilization warningunit 730 may warn in real-time if the measured stabilization relatedindex exceeds a predetermined threshold.

The stabilization related index according to an exemplary embodiment ofthe present inventive concept may include at least one of a size of amargin area occurring through a video stabilization operation, amagnitude of a video camera motion sensed by a sensor, and a magnitudeof an inter-frame camera motion.

The stabilization index measurer 720 may measure the stabilizationrelated index for at least one of an inter-frame camera motionmeasurement operation, a camera view direction adjustment angledetermination operation, and a camera view direction compensationoperation by sensing the video camera stabilization operation of thevideo stabilization device 710 in real-time.

Accordingly, the video stabilization device 710 may sense the videocamera stabilization operation in real-time and warn a user thatexcessive video stabilization is performed if a video stabilizationindex exceeds a dangerous level in each operation.

Video stabilization methods for compensating for camera shaking by usinga relative inter-frame camera rotation angle as camera motioninformation has been described with reference to the respective videostabilization apparatuses 100 and 400 of FIGS. 1 and 4, and videostabilization methods for adjusting a camera view direction to beoriented to a predetermined target area by adjusting only the cameraview direction on a camera motion path has been described with referenceto the respective video stabilization apparatuses 200, 300, and 400 ofFIGS. 2, 3, and 4. Camera motion compensation and video stabilizationsimilar to vision shaking perceived by a human being may be implementedin a high speed by the video stabilization methods according to variousexemplary embodiments of the present inventive concept.

When a user A captures a video with a camera while running, since thecamera shakes or moves while capturing the video, a user B watching thevideo captured with the shaking camera may feel severely dizzy. However,even though the user A also shakes or moves while running, the user Adoes not feel so dizzy due to images visually perceived.

This is because a view of the user A is oriented to a subject on whichthe user A keeps focusing even though a position of eyes of the user Arapidly varies according to a motion of the user A. That is, since aview direction is continuously fixed to the focused subject so that thefocused subject is continuously located on the center of imagesperceived by the eyes of the user A, even though the position of theeyes of the user A continuously moves, the user A does not feel dizzyregardless of the body shaking of the user A.

As such, a motion of the eyes of a human being, which continuouslychanges a view direction of a user's eyes when a position of the eyescontinuously moves, will be applied to a camera. Accordingly, the videostabilization apparatuses 100, 200, 300, 400, 500, 600, and 700according to various exemplary embodiments of the present inventiveconcept may realize video stabilization similar to human sight byadjusting a camera view direction to be oriented to a predeterminedfocused subject according to a camera motion.

In order to do this, exemplary embodiments according to the presentinventive concept provide a video stabilization scheme of compensatingfor a camera motion by compensating for a camera view direction by adifference between a camera view direction of a current frame and auser's predicted view direction by using only a user's view direction ona camera motion path. In addition, the exemplary embodiments provide amethod of predicting only a camera view direction on a camera motionpath by measuring only a relative camera view direction, i.e., arotation angle of the camera view direction, in a camera motion throughadjustment of only the camera view direction.

A method of predicting a relative camera motion between sequentialframes will be described below. When view directions of a Kth image anda (K+1)th image are different from each other, correct view directionsof the Kth image and the (K+1)th image do not have to be known to matchthe view directions of the Kth image and the (K+1)th image. Instead, ifonly a relative direction difference, i.e., an inter-frame viewdirection difference, between view vectors of the Kth image, whichcorresponds to a reference view direction, and the (K+1)th image isknown, the view direction of the (K+1)th image may be rotated by theinter-frame view direction difference to match the view direction of theKth image. Thus, any one of the video stabilization apparatuses 100,200, 300, 400, 500, 600, or 700 according to exemplary embodiments ofthe present inventive concept may be used to predict only a relativeview vector difference between two frames.

In addition, a camera view direction may be adjusted by predicting auser's view direction using a relative camera motion and measuring adifference between a camera view direction of a current frame and theuser's predicted view direction. Matching view directions of N frames toa single view direction may be performed by changing the view directionsof the N frames based on a reference view direction. Even though thereference view direction is not correctly defined in a three dimensionalspace, a camera motion during a frame sequence of the N frames may bepredicted by using only relative direction differences between the Nframes and the reference view direction.

For example, for the N frames, relative view directions, i.e., (N−1)view direction differences, between sequential frames may be obtained,and an average of the (N−1) view direction differences may bedetermined. The average of the (N−1) view direction differences may bedefined as a final reference view direction of the N frames. The finalreference view direction may also be defined by a relative rotationangle based on an initial view direction like view vectors of the Nframes. Thus, the final reference view direction may be a user's viewdirection or changed to another predetermined target view direction.

In addition, camera shaking may be compensated for to be similar tohuman sight by adjusting only a camera view direction by a differenceangle between the final reference view direction and a current cameraview direction without changing a camera position.

Camera motion measuring methods according to various exemplaryembodiments of the present inventive concept by which a relativeinter-frame camera rotation angle is measured will be described withreference to FIGS. 8 and 9.

FIG. 8 is a diagram for describing a camera motion measuring methodaccording to an exemplary embodiment of the present inventive concept.

To correctly obtain a relationship between two images captured indifferent general positions, information regarding a distance betweentwo center points of a camera and directions of camera view vectors isnecessary. However, a method of measuring a camera motion by determininga displacement or coordinate difference after acquiring correctpositions and coordinates of a subject of the two images by using theinformation regarding the distance between the two center points of thecamera and the directions of the camera view vectors requires aconsiderable amount of computation.

However, by employing a camera motion measuring method according to anexemplary embodiment of the present inventive concept, a relative cameramotion may be measured by measuring a relative direction between twocamera view directions, i.e., a rotation angle of the other camera viewdirection based on a reference camera view direction from among the twocamera view directions. In addition, a difference between the two cameraview directions of the camera may be determined in each of the X-axis,Y-axis, and Z-axis directions.

For example, a subject located in a current position 810 may be pickedup at an image center 860 by changing a camera view direction 820oriented to the current position 810 in an original image 800, which isa Kth image prior to an adjustment, to a reference camera view direction830 oriented to the image center 860 in image 850, which is the Kthimage after the adjustment.

Since a camera motion between successive actual images is not greatcomparing to a distance between a camera and a captured subject, amoving distance according to the camera motion may be approximatelyequal to zero. In addition, a trigonometric function may be used tomeasure a relative angle between predetermined points in images througha position relationship between an image center and the predeterminedpoints. However, since a rotation component in a camera motion betweensuccessive images is actually not great, the trigonometric function maybe approximately equal to a general polynomial expression. Assumingcharacteristics according to such a camera motion, a rotation angle ofthe camera may be simply described by a general polynomial expression byusing coordinates of predetermined points in images, and relative anglevariations in the X-axis and Y-axis directions may be derived as resultvalues.

Thus, a rotation angle θ between the current camera view direction 820and the reference camera view direction 830 may be predicted using acentral coordinate m of the current position 810 and a centralcoordinate m′ of the image center 860. In addition, if a displacementbetween m and m′ has both an X-axis directional component and a Y-axisdirectional component, a rotation angle in the X-axis direction and arotation angle in the Y-axis direction may be predicted according to adisplacement in the X-axis direction and a displacement in the Y-axisdirection in a frame, respectively.

FIG. 9 is a diagram for describing a method of measuring a camerarotation angle in the Z-axis direction, according to an exemplaryembodiment of the present inventive concept.

A rotation angle with respect to the Z-axis may be measured by usingstraight-lined motions of a plurality of sample points in an image. Forexample, straight-lined motions may occur on four sample points A, B, C,and D by a rotation of an image 900 in the Z-axis direction.

Due to the same rotation, the sample points A and C have straight-linedmotions 910 and 930, respectively, in directions opposite to each other,and the sample points B and D have straight-lined motions 920 and 940,respectively, in directions opposite to each other. For example, when aclockwise rotation occurs with respect to the Z-axis, the sample pointsA, B, C, and D may have straight-lined motions to the right, downwards,to the left, and upwards, respectively.

If a width and a height of the image 900 are different from each other,the magnitudes of straight-lined motions of the sample points A and Care different from the magnitudes of straight-lined motions of thesample points B and D. For example, if the width and the height of theimage 900 are W and H, respectively, and if the image 900 rotates by arotation angle θ, the sample points B and D have a quantity of motion ofapproximately W/2×θ, and the sample points B and D have a quantity ofmotion of approximately H/2×θ. Thus, a quantity of straight-lined motiondue to a rotary motion at each of the sample points A, B, C, and D maybe determined based on the width and the height of the image 900.

Thus, straight-lined motions, each having a predetermined magnitude anddirection, may occur at the sample points A, B, C, and D due to the samerotation, and magnitudes and directions of motion vectors instraight-lined directions, which occur at the sample points A, B, C, andD in correspondence with a rotation angle, may be determined based on adistance and direction from an original point to each of the samplepoints A, B, C, and D on a Z-axis plane.

After determining a cost value, such as a Sum of Absolute Difference(SAD), at each matching point in a current frame and a reference framefor each of the sample points A, B, C, and D, a motion vector having amagnitude and direction to minimize a final cost value based on costvalues of all of the sample points A, B, C, and D may be determined as amotion vector between the current frame and the reference frame. Thatis, a quantity of motion of the motion vector to minimize the final costvalue obtained by determining and summing the cost values of the samplepoints A, B, C, and D may be a final quantity of straight-lined motion.A measure of a rotation angle with respect to the Z-axis may bedetermined by converting the quantity of straight-lined motion to arotary motion.

The number of sample points, e.g., the sample points A, B, C, and D, formeasuring a quantity of motion may be increased or decreased. To measurea correct motion at each sample point, a method of measuring a motionvector in a sample area by selecting the sample area around each samplepoint may be used.

The inter-frame camera motion measurer 110 or 410 may determinemagnitudes and directions of motion vectors in straight-lineddirections, which occur in sample areas on a Z-axis plane incorrespondence with each of the rotation angles in a predeterminedrange, to measure a Z-axis direction camera motion between frames. Atotal cost value of the sample areas may be determined by measuring andsumming a cost value based on a difference (c=a−b) between an image amoved by a motion vector in a corresponding straight-lined direction anda corresponding sample area b of a frame in a reference camera referenceview direction for every sample area.

The inter-frame camera motion measurer 110 or 410 may detect a rotationangle for generating an initial total cost value by comparing totalcosts of the sample areas, which are determined for the rotation angles,and determine the detected rotation angle as a camera rotation angle inthe Z-axis direction.

Thus, the inter-frame camera motion measurer 110 or 410 may predict onlya camera rotation angle without predicting an actual camera position anddisplacement in a camera motion between frames and transmit informationregarding the predicted camera rotation angle between the frames to thecamera parameter determiner 120 or the camera view direction adjustmentangle determiner 220, 320, or 420. Accordingly, a computation amount formeasuring a correct displacement of a camera motion may be reduced froma computation amount for measuring the camera motion, and an informationamount to be transmitted to the camera parameter determiner 120 or thecamera view direction adjustment angle determiner 220, 320, or 420 mayalso be reduced.

FIG. 10 is a diagram showing a camera motion path, and FIG. 11 is adiagram for describing a conventional video stabilization method.

A camera motion path 1000 may be obtained by using a video acquired by auser capturing the video while moving with a camera. Informationregarding the camera moving along the camera motion path 1000 may bedefined by a camera position and a camera view direction in each frame.

A global camera motion path 1030 derived from the camera motion path1000 may be determined by analyzing a frame sequence of the cameramotion path 1000. In addition, each of actual camera view directions1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018 generated inframes on the camera motion path 1000 may be divided into a globalcamera view direction based on the global camera motion path 1030 and alocal camera view direction generated in a corresponding frame.

In detail with two frames on the camera motion path 1000, the actualcamera view directions 1002 and 1004 may be divided into global cameraview directions 1032 and 1034 based on the global camera motion path1030 and local camera view directions, respectively.

According to a conventional video stabilization method, a method ofpredicting actual camera positions and view directions on the cameramotion path 1000 to match the global camera motion path 1030 may beused.

That is, referring also to FIG. 11, after measuring the actual cameraview directions 1002 and 1004 and the global camera view directions 1032and 1034 based on original points of respective coordinate systems,respective angular differences θ2 and θ4 between the actual camera viewdirections 1002 and 1004 and the global camera view directions 1032 and1034 are determined. The actual camera view directions 1002 and 1004 maymatch the global camera view directions 1032 and 1034 by rotating by θ2and θ4, respectively.

In addition, actual camera positions of the actual camera viewdirections 1002 and 1004 on the camera motion path 1000 and theircorresponding positions on the global camera motion path 1030 arepredicted. Respective translational differences D2 and D4 between theactual camera positions of the actual camera view directions 1002 and1004 and their corresponding positions on the global camera motion path1030 are determined, respectively. The actual camera positions of theactual camera view directions 1002 may match their correspondingpositions on the global camera motion path 1030 by moving by D2 and D4,respectively.

Accordingly, based on the conventional video stabilization methoddescribed above, an actual camera motion path must be adjusted to matcha global camera motion path by correctly measuring all camera positionsand view directions on a camera motion path and camera positions andview directions on the global camera motion path.

The video stabilization apparatuses 200, 300, and 400 may compensate forcamera shaking by predicting a user's view direction through generationof a camera motion path and the use of only a difference between acamera motion path of a current frame and the user's predicted viewdirection. Camera shaking compensation methods according to variousexemplary embodiments of the present inventive concept, by which acamera view direction is adjusted to be oriented to a predetermined viewdirection, will be described in detail with reference to FIGS. 12 and13.

FIG. 12 is a diagram showing a camera motion path obtained by adjustinga camera motion based on a global camera view direction of the cameramotion path, according to an exemplary embodiment of the presentinventive concept. FIG. 13 is a diagram showing a camera motion pathobtained by adjusting a camera motion based on a camera view directionoriented to a target area, according to an exemplary embodiment of thepresent inventive concept.

The camera view direction adjustment angle determiner 220, 320, or 420of the respective video stabilization apparatuses 200, 300, or 400 maygenerate the camera motion path 1000 by using camera rotation angleinformation acquired from the camera motion measurer 210 or 310 or theinter-frame camera motion measurer 410. Current camera view directions1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018 of frames alongthe camera motion path 1000 may be defined.

Since the camera view direction adjustment angle determiner 220, 320, or420 aims at not moving an actual position of a camera but adjusting acamera view direction to be fixed onto a predetermined focal point, thecamera view direction adjustment angle determiner 220, 320, or 420requires only a relative rotation angle of the camera view directionbased on a predetermined direction without predicting a correct cameraposition on a camera motion path.

That is, the camera view direction adjustment angle determiner 220, 320,or 420 may determine the global camera motion path 1030 occurring alongthe camera motion path 1000 by analyzing a frame sequence on the cameramotion path 1000, and the current camera view directions 1002, 1004,1006, 1008, 1010, 1012, 1014, 1016, and 1018 generated in correspondingframes on the camera motion path 1000 may be respectively divided intoglobal camera view directions 1032, 1034, 1036, 1038, 1040, 1042, 1044,1046, and 1048 and local camera view directions generated in thecorresponding frames.

The camera view direction adjustment angle determiner 220, 320, or 420may determine camera view direction adjustment angles based on thecurrent camera view directions 1002, 1004, 1006, 1008, 1010, 1012, 1014,1016, and 1018 and the global camera view directions 1032, 1034, 1036,1038, 1040, 1042, 1044, 1046, and 1048 on the global camera motion path1030. The camera view direction adjustment angle determiner 220, 320, or420 may transmit information regarding a camera view directionadjustment angle determined for each of the frames to the camera viewdirection compensator 230, 330, or 430.

In particular, since a camera motion path generated according to theembodiment described with reference to FIG. 4 is generated based on onlyinter-frame camera rotation angle information, correct inter-framecamera positions and correct inter-frame camera capturing angles on thecamera motion path are not determined, and rotation angles of cameraview directions along the camera motion path may be continuouslydetermined. Thus, each of the current camera view directions 1002, 1004,1006, 1008, 1010, 1012, 1014, 1016, and 1018 determined by the cameraview direction adjustment angle determiner 420 may correspond to arotation angle based on a predetermined reference camera view direction.Likewise, each of the global camera view directions 1032, 1034, 1036,1038, 1040, 1042, 1044, 1046, and 1048, the local camera viewdirections, and the camera view direction adjustment angles oriented toa predetermined view direction, which are determined by the camera viewdirection adjustment angle determiner 420, may also be defined by arelative direction difference, i.e., a rotation angle, based on thereference camera view direction.

The camera view direction compensator 230, 330, or 430 may compensatefor a camera motion by adjusting a current camera view direction by acamera view direction adjustment angle based on the camera viewdirection adjustment angle information received from the camera viewdirection adjustment angle determiner 220, 320, or 420.

For example, the camera view direction compensator 230, 330, or 430 mayrotate the respective current camera view directions 1002, 1004, 1006,1008, 1010, 1012, 1014, 1016, and 1018 on the camera motion path 1000 bytheir corresponding camera view direction adjustment angles in theircorresponding frames.

As illustrated in FIG. 12, the camera view direction adjustment angledeterminer 220, 320, or 420 may determine the camera view directionadjustment angle information based on the global camera view directions1032, 1034, 1036, 1038, 1040, 1042, 1044, 1046, and 1048. The cameraview direction adjustment angle determiner 220, 320, or 420 may measuredifference angles between the respective current camera view directions1002, 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018 on the cameramotion path 1000 and their corresponding global camera view directions1032, 1034, 1036, 1038, 1040, 1042, 1044, 1046, and 1048 and determinethe measured difference angles as camera view direction adjustmentangles. That is, the camera view direction compensator 230, 330, or 430may rotate the respective current camera view directions 1002, 1004,1006, 1008, 1010, 1012, 1014, 1016, and 1018 by their correspondingcamera view direction adjustment angles in their corresponding frames tomatch their corresponding global camera view directions 1032, 1034,1036, 1038, 1040, 1042, 1044, 1046, and 1048.

Thus, only the current camera view directions 1002, 1004, 1006, 1008,1010, 1012, 1014, 1016, and 1018 may be adjusted to their correspondingcamera view directions 1052, 1054, 1056, 1058, 1060, 1062, 1064, 1066,and 1068 by the camera view direction compensator 230, 330, or 430without a position change in their corresponding frames on the cameramotion path 1000. That is, after the adjustment, the camera viewdirections 1052, 1054, 1056, 1058, 1060, 1062, 1064, 1066, and 1068 maymatch the corresponding global camera view directions 1032, 1034, 1036,1038, 1040, 1042, 1044, 1046, and 1048 in their current frames,respectively.

As illustrated in FIG. 13, the camera view direction adjustment angleinformation may be determined so that camera view directions areoriented to a predetermined target point. The camera view directionadjustment angle determiner 220, 320, or 420 may measure a differenceangle between a current camera view direction on the camera motion path1000 and a target view direction oriented to a target point 1330 anddetermine the measured difference angle as a camera view directionadjustment angle. That is, the camera view direction compensator 230,330, or 430 may rotate the current camera view directions 1002, 1004,1006, 1008, 1010, 1012, 1014, 1016, and 1018 by their correspondingcamera view direction adjustment angles in their corresponding frames tomatch their corresponding camera view directions 1332, 1334, 1336, 1338,1340, 1342, 1344, 1346, and 1348 oriented to the target point 1330.

Thus, the camera view directions 1332, 1334, 1336, 1338, 1340, 1342,1344, 1346, and 1348 for which camera motions are compensated by thecamera view direction compensator 230, 330, or 430 may be adjusted tofocus on the target point 1330 in their corresponding current positions.

Accordingly, the video stabilization apparatuses 200, 300, and 400 maynot only adjust a global camera view direction on a global camera motionpath or a camera view direction oriented to a frame center, but alsoadjust a camera view direction to be oriented to a predetermined targetpoint or fixed to a target view direction. The predetermined targetpoint to which the camera view direction is oriented or the target viewdirection may be set according to a desire of a user capturing a video.

In addition, since a video stabilization operation may be performedwhile reproducing a video by using a video stabilization methodaccording to an exemplary embodiment of the present inventive concept innot only a video capturing device but also a video reproducing device, apredetermined target point to which a camera view direction is orientedor a target view direction may be set according to a desire of a userwatching the video.

Since a camera view direction is fixed to be continuously oriented to apredetermined target point without changing an actual camera motion pathand a camera position, the video stabilization apparatuses 200, 300, and400 may acquire a video similar to a motion visually perceived by ahuman being. In addition, since a camera motion may be compensated forby adjusting a camera view direction by using only the camera viewdirection without measuring a correct displacement on a camera motionpath or using only a relative variation or rotation angle of the cameraview direction, the camera motion path may be correctly recovered,thereby reducing a computation amount as compared with conventionalmethods of adjusting a camera displacement and a camera view directionby correctly recovering a camera motion path.

User interfaces for effectively using video stabilization methods of thevideo stabilization apparatuses 100, 200, 300, and 400 will be describedbelow with reference to FIGS. 14 to 16.

FIG. 14 is a diagram showing user interfaces for adjusting a size of amargin area and a degree of video stabilization according to a videostabilization operation, according to an exemplary embodiment of thepresent inventive concept.

The camera view direction compensator 130, 230, 330, or 430 of therespective video stabilization apparatus 100, 200, 300, or 400 mayrotate a current camera view direction to a target view direction. Inparticular, a camera view direction in the Z-axis direction may bechanged by rotating an image in parallel to the Z-axis plane by arotation angle in the Z-axis direction among rotation angle componentsin the X-axis, Y-axis, and Z-axis directions of a camera view directionadjustment angle.

When video stabilization is performed according to such a rotation of acamera view direction, since the outer portion of an image is lost ornot completely recovered due to a rotation of a frame, an angle of viewactually seen by a user is narrower than the angle of view correspondingto an original image. As a result, a user of a video reproducing devicemay watch only an image within the angle of view, which remains in theinner portion, or only a valid image recovered in such a way may bestored in a video capturing device or a storage device. The higher adegree of video stabilization is, the greater amount of an area (i.e.,margin area) is lost according to the extent of the increase in videostabilization. If the video stabilization is excessively performedwithout considering a margin area, a size of a valid image may beconsiderably reduced. That is, a trade-off may exist between a videostabilization effect and a margin area size.

Thus, any of the video stabilization apparatuses 100, 200, 300, 400,500, or 700 may be equipped in the same manner as the videostabilization device 610 of the video stabilization apparatus 600. Themargin area user input unit 620 of the video stabilization apparatus 600may provide user interfaces 1400, 1420, 1440, and 1460 for a user totune a margin area and a degree of video stabilization.

The user interface 1400 displays a slice bar for the user to freelyselect a size of a margin area or a ratio of the size of the margin areato the entire image size. The margin area user input unit 620 accordingto the exemplary embodiments of FIGS. 1 to 4 may determine a degree ofvideo stabilization based on the size of the margin area or the ratio ofthe size of the margin area to the entire image size, which is inputthrough the user interface 1400.

The user interface 1420 displays a slice bar for the user to freelyselect a video stabilization level indicating a degree of videostabilization. The margin area user input unit 620 according to theexemplary embodiments of FIGS. 1 to 4 may determine a size of a marginarea or a ratio of the size of the margin area to the entire image sizebased on the video stabilization level inputted through the userinterface 1420.

The user interface 1440 displays a slice bar for the user to freelyselect a trade-off of a video stabilization level in comparison with amargin area. The user interface 1460 displays a lookup table of videostabilization levels in comparison with margin areas for the user toselect a combination in which a trade-off of a video stabilization levelin comparison with a margin area is optimized. The user may select amargin area size, a video stabilization level, or a trade-off of a videostabilization level in comparison with a margin area by using a usercontroller linked to the user interfaces 1400, 1420, 1440, and 1460.

The margin area user input unit 620 according to the exemplaryembodiments of FIGS. 1 to 4 may determine a margin area size or a degreeof video stabilization based on a user's input through the userinterface 1400, and the camera parameter determiner 120 or the cameraview direction adjustment angle determiner 220, 320, or 420 maydetermine a camera view direction adjustment angle based on thedetermined margin area size or the determined degree of videostabilization.

FIG. 15 is a diagram showing a user interface for a user to set a targetarea of a camera view direction for a video stabilization operation,according to an exemplary embodiment of the present inventive concept.

Any one of the video stabilization apparatuses 300, 400, 600, or 700 maybe equipped in the same manner as the video stabilization device 510 ofthe video stabilization apparatus 500. The view direction target areauser input unit 520 of the video stabilization apparatus 500 may providea user interface 1500 for a user to set a target area of a camera viewdirection or a target view direction. The user interface 1500 displays aselection window 1510 capable of freely moving left and right and up anddown on a current video capturing scene or a captured video image. Theuser may move the selection window 1510 to a desired target point towhich a camera view direction is fixed by using a motion controller ofthe selection window 1510, which is linked to the user interface 1500.

The view direction target area determiner 315 or 415 according to therespective exemplary embodiments of FIGS. 3 and 4 may determine a targetarea and a target view direction based on a user's input through theuser interface 1500, and the camera view direction adjustment angledeterminer 320 or 420 may determine a camera view direction adjustmentangle.

FIG. 16 is a diagram showing a user interface for warning an excessivevideo stabilization operation while performing a video stabilizationoperation, according to an exemplary embodiment of the present inventiveconcept.

Any one of the video stabilization apparatuses 100, 200, 300, 400, 500,or 600 may be equipped in the same manner as the video stabilizationdevice 710 of the video stabilization apparatus 700. The stabilizationindex measurer 720 of the video stabilization apparatus 700 may measurea degree of video stabilization in each stage while performing a videostabilization operation. The stabilization index measurer 720 maymeasure a stabilization related index by sensing the degree of videostabilization in real-time during the video stabilization operation.

The video stabilization apparatus 700 may measure a stabilizationrelated index, such as a margin area size generated through a videostabilization operation, a magnitude of a video camera motion sensedthrough a sensor, or a magnitude of an inter-frame camera motion, inreal-time. For example, since a camera view direction adjustment angle,which is camera motion information to be compensated for, variesproportionally as a difference between a camera motion path and anaccumulated inter-frame camera view direction, which is generated by thevideo stabilization apparatus 700, varies, excessive video stabilizationmay be predicted. Accordingly, the camera view direction adjustmentangle may be used as a stabilization-related index.

The stabilization index measurer 720 may sense a stabilization-relatedindex in real-time in a video stabilization operation of the videostabilization device 710. For example, the stabilization-related indexmay be measured in each of an inter-frame camera motion measuring stage,a camera view direction adjustment angle determining stage, and a cameraview direction compensating stage of a video stabilization operation.The video stabilization warning unit 730 may warn in real-time or guidea user about a current stabilization state if the stabilization-relatedindex measured in each of the inter-frame camera motion measuring stage,the camera view direction adjustment angle determining stage, and thecamera view direction compensating stage of the video stabilizationoperation exceeds a predetermined threshold. The real-time warning maybe implemented by a warning window 1610 displayed on a user interface1600 or an audio warning sound 1620.

For example, if a camera motion sensed by the stabilization indexmeasurer 720 exceeds the threshold during an inter-frame camera motionmeasuring operation, the video stabilization warning unit 730 may warnin real-time. In addition, if a camera view direction adjustment anglesensed by the stabilization index measurer 720 exceeds the thresholdduring a camera view direction adjustment angle determining operation,the video stabilization warning unit 730 may warn in real-time. Inaddition, if a margin area sensed by the stabilization index measurer720 exceeds the threshold during a camera view direction compensatingoperation, the video stabilization warning unit 730 may warn inreal-time.

In addition, if the video stabilization warning unit 730 can predict amargin area based on camera motion information and camera view directionadjustment angle information in the inter-frame camera motion measuringstage and the camera view direction adjustment angle determining stageduring the video stabilization operation, the video stabilizationwarning unit 730 may warn in real-time when the margin area exceeds thethreshold during all of the video stabilization operation.

A user may predict a degree of a user's motion according to guidance orwarning based on a video stabilization state of the video stabilizationapparatus 700. That is, if there is guidance or warning that a videostabilization state is excessive, since it is analyzed that a user'scamera motion requiring video stabilization is great, the user mayprevent an excessive camera motion.

The threshold of the stabilization-related index may be determinedwithin a computation amount which the video stabilization apparatus 700can stably process by analyzing a current system and video. Thethreshold of the stabilization related index may be determined based ona trade-off of a margin area in comparison with a degree of videostabilization, which is set by the user.

FIG. 17 is a flowchart of a video stabilization method according to anexemplary embodiment of the present inventive concept.

Referring to FIG. 17, in operation 1710, a difference angle of a currentcamera view direction in comparison with a reference camera viewdirection is determined in each frame of a frame sequence of a video,and an inter-frame camera motion is measured based on the relativedifference angle of the current camera view direction in comparison withthe reference camera view direction.

In operation 1720, a camera motion path of the frame sequence isgenerated by using the inter-frame camera motion, and a camera parameterin each frame is determined by using the camera motion path. The cameraparameter may include camera displacement and/or camera directioncompensation values on the camera motion path for camera motioncompensation.

In operation 1730, a camera motion is compensated for by using thecamera parameter in each frame.

The video stabilization method according to the exemplary embodiment ofFIG. 17 corresponds to the video stabilization scheme of compensatingfor a camera motion by extracting only relative rotation angleinformation relating to inter-frame camera motion in the videostabilization apparatus 100 according to the exemplary embodiment ofFIG. 1.

FIG. 18 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 18, in operation 1810, a camera motion is measured ineach frame of a frame sequence of a video. In operation 1820, a cameramotion path of the frame sequence is generated by using the cameramotion in each frame, and a camera view direction adjustment anglebetween a user's view direction and a camera view direction isdetermined by using the camera motion path. In operation 1830, thecamera view direction is compensated for by using the camera viewdirection adjustment angle in each frame.

The video stabilization method according to the exemplary embodiment ofFIG. 18 corresponds to the video stabilization scheme of compensatingfor a camera motion by adjusting only a camera view direction to auser's view direction on a camera motion path in the video stabilizationapparatus 200 according to the exemplary embodiment of FIG. 2.

FIG. 19 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 19, in operation 1910, a camera motion is measured ineach frame of a frame sequence of a video. In operation 1920, a targetarea of a view direction in the frame sequence is determined. Inoperation 1930, a camera motion path of the frame sequence is generatedby using the camera motion in each frame, and a camera view directionadjustment angle based on a view direction oriented to the target areais determined by using the camera motion path. In operation 1940, acamera view direction is compensated for by using the camera viewdirection adjustment angle in each frame.

The video stabilization method according to the exemplary embodiment ofFIG. 19 corresponds to the video stabilization scheme of compensatingfor a camera motion by adjusting only a camera view direction to a viewdirection oriented to a target area on a camera motion path in the videostabilization apparatus 300 according to the exemplary embodiment ofFIG. 3.

FIG. 20 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 20, in operation 2010, a difference angle of a currentcamera view direction in comparison with a reference camera viewdirection is determined in each frame of a frame sequence of a video,and an inter-frame camera motion is measured based on the relativedifference angle of the current camera view direction in comparison withthe reference camera view direction. In operation 2020, a camera motionpath of the frame sequence is generated by using the inter-frame cameramotion, and a camera view direction adjustment angle based on a user'sview direction is determined by using the camera motion path. Inparticular, the user's view direction may be set to a camera viewdirection oriented to a target area according to a user's input.

In operation 2030, a camera view direction is compensated for by usingthe camera view direction adjustment angle in each frame.

The video stabilization method according to the exemplary embodiment ofFIG. 20 corresponds to the video stabilization scheme of compensatingfor a camera motion by extracting only relative rotation angleinformation according to an inter-frame camera motion and adjusting onlya camera view direction to a view direction oriented to a target area ona camera motion path in the video stabilization apparatus 400 accordingto the exemplary embodiment of FIG. 4.

FIG. 21 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 21, in operation 2110, a selection request of a targetarea of a view direction in a frame sequence is received from a userthrough a user interface. In operation 2120, a difference angle of acurrent camera view direction in comparison with a reference camera viewdirection is determined in each frame of a frame sequence of a video,and an inter-frame camera motion is measured based on the relativedifference angle of the current camera view direction in comparison withthe reference camera view direction.

In operation 2130, a camera motion path of the frame sequence isgenerated by using the inter-frame camera motion, and a camera viewdirection adjustment angle based on a view direction oriented to atarget area is determined by using the camera motion path. In operation2140, a camera view direction is compensated for by using the cameraview direction adjustment angle in each frame.

The video stabilization method according to the exemplary embodiment ofFIG. 21 corresponds to the video stabilization scheme of compensatingfor a camera motion by considering a target area of a view directionaccording to a user's input in the video stabilization apparatus 500according to the exemplary embodiment of FIG. 5. Accordingly, the videostabilization method according to the exemplary embodiment of FIG. 21may conform to the video stabilization methods according to theexemplary embodiments of FIGS. 17 to 20.

FIG. 22 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 22, in operation 2210, a selection request of at leastone of a margin area size and a stabilization level for videostabilization is received from a user through a user interface.

In operation 2220, a video stabilization operation for stabilizing avideo of a camera is performed by measuring a relative inter-framecamera motion based on a reference camera view direction in a framesequence of a video based on the at least one of the margin area sizeand the stabilization level, determining a camera view directionadjustment angle based on a view direction oriented to a target area,and compensating for a camera view direction by using the camera viewdirection adjustment angle in each frame. The video stabilizationoperation in operation 2220 may conform to the video stabilizationmethods according to the exemplary embodiments of FIGS. 17 to 20.

The video stabilization method according to the exemplary embodiment ofFIG. 22 corresponds to the video stabilization scheme of compensatingfor a camera motion by considering a margin area according to a user'sinput in the video stabilization apparatus 600 according to theexemplary embodiment of FIG. 6.

FIG. 23 is a flowchart of a video stabilization method according toanother exemplary embodiment of the present inventive concept.

Referring to FIG. 23, in operation 2310, a video of a camera isstabilized by measuring a relative inter-frame camera motion based on areference camera view direction in a frame sequence of the video,determining a camera view direction adjustment angle based on a viewdirection oriented to a target area, and compensating for a camera viewdirection by using the camera view direction adjustment angle in eachframe.

In operation 2320, a stabilization-related index is measured whileperforming the stabilization of the video of the camera of operation2310. In operation 2330, warning is performed in real-time if thestabilization-related index exceeds a predetermined threshold.

The video stabilization method according to the exemplary embodiment ofFIG. 23 corresponds to the video stabilization scheme of the videostabilization apparatus 700 according to the exemplary embodiment ofFIG. 7.

Since conventional video stabilization methods based on two dimensional(2D) analysis compensate for an original image by using a simpleparallel movement scheme or a simple warping scheme, image distortionoccurs. In addition, since conventional 3D analysis-based methodsgenerate a new image by reconstructing a 3D image, a relatively correctimage can be generated. However, since a considerable amount ofcomputation is required to reconstruct the 3D image and perfectreconstruction is difficult, image distortion still occurs.

On the contrary, since each of the video stabilization apparatuses 100,200, 300, and 400 according to various exemplary embodiments of thepresent inventive concept and each of the video stabilization methods ofFIGS. 17 to 22 implements a 3D scheme by predicting a view vectordirection difference between images based on a conventional 2D analysisscheme and adjusting a camera view direction in each frame on a cameramotion path instead of simple parallel movement of a camera position, avideo having very low distortion in a point of view of human sight maybe outputted.

Exemplary embodiments of the present inventive concept can be written ascomputer programs and can be implemented in general-use digitalcomputers that execute the programs using a computer-readable recordingmedium. Examples of the computer-readable recording medium may includestorage media such as magnetic storage media (e.g., ROM, floppy disks,hard disks, etc.) and optical recording media (e.g., CD-ROMs, or DVDs).

While the present inventive concept has been particularly shown anddescribed with reference to exemplary embodiments thereof, it will beunderstood by those of ordinary skill in the art that various changes inform and details may be made therein without departing from the spiritand scope of the present inventive concept as defined by the appendedclaims. The exemplary embodiments should be considered in a descriptivesense only and not for purposes of limitation. Therefore, the scope ofthe present inventive concept is defined not by the detailed descriptionof the exemplary embodiments but by the appended claims, and alldifferences within the scope will be construed as being included in thepresent disclosure.

1. A video stabilization method comprising: measuring an inter-framecamera motion based on an angle of a relative camera view direction withrespect to a reference camera view direction in each frame of a framesequence of a video; generating a camera motion path of the framesequence by using the measured inter-frame camera motion and determininga camera parameter for each frame by using the generated camera motionpath; and compensating for a camera motion by using the determinedcamera parameter for each frame.
 2. The video stabilization method ofclaim 1, wherein the measuring of the inter-frame camera motioncomprises determining a difference angle of relative camera viewdirections between each pair of temporally successive frames in theframe sequence.
 3. The video stabilization method of claim 2, whereinthe measuring of the inter-frame camera motion further comprisescomputing an average of the difference angles of the relative cameraview directions between each pair of temporally successive frames in theframe sequence and using the computed average as the reference cameraview direction.
 4. The video stabilization method of claim 1, whereinthe measuring of the inter-frame camera motion comprises measuringrelative camera rotation angles in each of an X-axis direction, a Y-axisdirection, and a Z-axis direction.
 5. The video stabilization method ofclaim 4, wherein the measuring of the inter-frame camera motion furthercomprises determining a camera rotation angle in the Z-axis direction byusing straight-lined motions occurring in respective predetermined areason a Z-axis plane.
 6. The video stabilization method of claim 5, whereinthe determining of the camera rotation angle in the Z-axis directioncomprises: determining magnitudes and directions of straight-linedmotion vectors occurring in the respective predetermined areas on theZ-axis plane in correspondence with each of respective rotation angleswithin a predetermined range; determining a cost value based on an errorbetween an image moved by a straight-lined motion vector and acorresponding area of a frame in a reference camera reference viewdirection for each of the respective predetermined areas and summing thecost values of the respective predetermined areas; and detecting arotation angle generating a minimum total cost value by determining thesummed cost value of the respective predetermined areas for each of therespective rotation angles and comparing the summed cost values to eachother, and using the detected rotation angle as the camera rotationangle in the Z-axis direction.
 7. The video stabilization method ofclaim 6, wherein the magnitudes and directions of the straight-linedmotion vectors occurring in the respective predetermined areas incorrespondence with each of the respective rotation angles aredetermined based on respective distances and respective directions froman original point of the Z-axis plane to each of the correspondingpredetermined areas, respectively.
 8. The video stabilization method ofclaim 1, wherein the determining of the camera parameter comprises:generating the camera motion path of the frame sequence by accumulatinginformation relating to the inter-frame camera motion; generating aglobal camera motion path of the generated camera motion path; anddetermining a camera adjustment value for each frame with respect to theglobal camera motion path.
 9. The video stabilization method of claim 1,wherein the compensating for the camera motion comprises: rotating thecamera view direction by a view direction adjustment value relating tothe determined camera parameter; and moving a camera position by adisplacement adjustment value relating to the determined cameraparameter.
 10. The video stabilization method of claim 1, wherein themeasuring of the inter-frame camera motion comprises measuring aninter-viewpoint camera motion based on a difference angle of a remainingviewpoint camera view direction with respect to a reference viewpointcamera view direction determined by using a left-viewpoint frame and aright-viewpoint frame corresponding to each other in each frame of astereoscopic video.
 11. A video stabilization method comprising:measuring a camera motion in each frame of a frame sequence of a video;generating a camera motion path of the frame sequence by using themeasured camera motion in each frame and determining a camera viewdirection adjustment angle between a user's view direction and a cameraview direction by using the generated camera motion path; andcompensating for the camera view direction by using the determinedcamera view direction adjustment angle for each frame.
 12. The videostabilization method of claim 11, wherein the determining of the cameraview direction adjustment angle comprises: separating a global cameraview direction and a local camera view direction from the generatedcamera motion path; and determining a difference angle between theglobal camera view direction and the local camera view direction andusing the determined difference angle as the camera view directionadjustment angle.
 13. The video stabilization method of claim 11,wherein the determining of the camera view direction adjustment anglecomprises determining a camera rotation angle for moving a currentcamera view direction on the generated camera motion path to a framecenter and using the determined camera rotation angle as the camera viewdirection adjustment angle.
 14. The video stabilization method of claim11, wherein the compensating for the camera view direction comprisesrotating the camera view direction of each frame by the camera viewdirection adjustment angle.
 15. The video stabilization method of claim11, wherein the measuring of the camera motion in each frame comprisesmeasuring an inter-viewpoint camera motion based on a difference angleof a remaining viewpoint camera view direction with respect to areference viewpoint camera view direction determined by using aleft-viewpoint frame and a right-viewpoint frame corresponding to eachother in each frame of a stereoscopic video.
 16. A video stabilizationmethod comprising: measuring a camera motion in each frame of a framesequence of a video; determining a target area of a view direction inthe frame sequence; generating a camera motion path of the framesequence by using the measured camera motion in each frame anddetermining a camera view direction adjustment angle based on thedetermined target area of the view direction by using the generatedcamera motion path; and compensating for a camera view direction byusing the determined camera view direction adjustment angle for eachframe.
 17. The video stabilization method of claim 16, wherein thedetermining of the camera view direction adjustment angle comprisespredicting a user's view direction by using the generated camera motionpath.
 18. The video stabilization method of claim 17, wherein thedetermining of the camera view direction adjustment angle furthercomprises: separating a global camera view direction and a local cameraview direction from the generated camera motion path and using theglobal camera view direction as the user's view direction; determining adifference angle between the global camera view direction and the localcamera view direction and using the determined difference angle as afirst adjustment angle of the camera view direction; determining adifference angle between the global camera view direction and a cameraview direction corresponding to the target area in each frame as asecond adjustment angle of the camera view direction based on adisplacement between a point corresponding to the global camera viewdirection and the target area in each frame; and determining a sum ofthe first adjustment angle and the second adjustment angle of the cameraview direction and using the determined sum as the camera view directionadjustment angle.
 19. The video stabilization method of claim 16,wherein the compensating for the camera view direction comprisesrotating the camera view direction of each frame by the camera viewdirection adjustment angle.
 20. The video stabilization method of claim16, wherein the measuring of the camera motion in each frame comprisesmeasuring an inter-viewpoint camera motion based on a difference angleof a remaining viewpoint camera view direction with respect to areference viewpoint camera view direction determined by using aleft-viewpoint frame and a right-viewpoint frame corresponding to eachother in each frame of a stereoscopic video.
 21. A video stabilizationmethod comprising: measuring an inter-frame camera motion based on adifference angle of a relative camera view direction with respect to areference camera view direction in each frame of a frame sequence of avideo; generating a camera motion path of the frame sequence by usingthe measured inter-frame camera motion and determining a camera viewdirection adjustment angle based on a user's view direction by using thegenerated camera motion path; and compensating for the camera viewdirection by using the determined camera view direction adjustment anglefor each frame.
 22. The video stabilization method of claim 21, whereinthe measuring of the inter-frame camera motion comprises determining adifference angle of relative camera view directions between each pair oftemporally successive frames in the frame sequence.
 23. The videostabilization method of claim 22, wherein the measuring of theinter-frame camera motion comprises computing an average of thedifference angles of relative camera view directions between each pairof temporally successive frames in the frame sequence and using thecomputed average as the reference camera view direction.
 24. The videostabilization method of claim 21, wherein the measuring of theinter-frame camera motion comprises measuring relative camera rotationangles in each of an X-axis direction, a Y-axis direction, and a Z-axisdirection.
 25. The video stabilization method of claim 24, wherein themeasuring of the inter-frame camera motion further comprises determininga camera rotation angle in the Z-axis direction by using straight-linedmotions occurring in respective predetermined areas on a Z-axis plane.26. The video stabilization method of claim 25, wherein the determiningof the camera rotation angle in the Z-axis direction comprises:determining magnitudes and directions of straight-lined motion vectorsoccurring in the respective predetermined areas on the Z-axis plane incorrespondence with each of respective rotation angles within apredetermined range; determining a cost value based on an error betweenan image moved by a straight-lined motion vector and a correspondingarea of a frame in a reference camera reference view direction for eachof the respective predetermined areas and summing the cost values of therespective predetermined areas; and detecting a rotation anglegenerating a minimum total cost value by determining the summed costvalue of the respective predetermined areas for each of the respectiverotation angles and comparing the summed cost values to each other, andusing the detected rotation angle as the camera rotation angle in theZ-axis direction.
 27. The video stabilization method of claim 26,wherein the magnitudes and directions of the straight-lined motionvectors occurring in the respective predetermined areas incorrespondence with each of the respective rotation angles aredetermined based on respective distances and respective directions froman original point of the Z-axis plane to each of the correspondingpredetermined areas, respectively.
 28. The video stabilization method ofclaim 21, wherein the determining of the camera view directionadjustment angle comprises generating the camera motion path of theframe sequence by accumulating information relating to the inter-framecamera motion.
 29. The video stabilization method of claim 28, whereinthe determining of the camera view direction adjustment angle furthercomprises predicting a user's view direction by using the generatedcamera motion path.
 30. The video stabilization method of claim 29,wherein the determining of the camera view direction adjustment anglefurther comprises: separating a global camera view direction and a localcamera view direction from the generated camera motion path and usingthe global camera view direction as the user's view direction; anddetermining the camera view direction adjustment angle based on theglobal camera view direction at each point on the generated cameramotion path.
 31. The video stabilization method of claim 30, wherein thedetermining of the camera view direction adjustment angle comprises:determining a target area of a view direction in the frame sequence; andgenerating the camera motion path of the frame sequence by using themeasured inter-frame camera motion and determining the camera viewdirection adjustment angle based on the determined target area of theview direction.
 32. The video stabilization method of claim 31, whereinthe determining of the target area of the view direction comprisessetting the global camera view direction on the generated camera motionpath to a view direction corresponding to the target area.
 33. The videostabilization method of claim 21, wherein the compensating for thecamera view direction comprises rotating the camera view direction bythe camera view direction adjustment angle in each frame.
 34. The videostabilization method of claim 21, wherein the measuring of theinter-frame camera motion comprises measuring an inter-viewpoint cameramotion based on a difference angle of a remaining viewpoint camera viewdirection with respect to a reference viewpoint camera view directiondetermined by using a left-viewpoint frame and a right-viewpoint framecorresponding to each other in each frame of a stereoscopic video.
 35. Avideo stabilization method comprising: receiving a selection request fora target area of a view direction in a frame sequence from a userthrough a user interface; measuring an inter-frame camera motion basedon a difference angle of a relative camera view direction with respectto a reference camera view direction in each frame of a frame sequenceof a video; generating a camera motion path of the frame sequence byusing the measured inter-frame camera motion and determining a cameraview direction adjustment angle based on the view directioncorresponding to the target area by using the generated camera motionpath; and compensating for the camera view direction by using the cameraview direction adjustment angle for each frame.
 36. A videostabilization method comprising: receiving a selection request for atleast one of a margin area size and a stabilization level for videostabilization from a user through a user interface; and stabilizing avideo of a camera by measuring a relative inter-frame camera motionbased on a reference camera view direction in a frame sequence of thevideo based on the at least one of the margin area size and thestabilization level, determining a camera view direction adjustmentangle based on a view direction corresponding to a target area, andcompensating for a camera view direction by using the determined cameraview direction adjustment angle for each frame.
 37. The videostabilization method of claim 36, wherein the stabilizing of the videoof the camera comprises: measuring a camera motion in each frame of theframe sequence of the video; determining a user's view direction on acamera motion path by generating the camera motion path of the framesequence using the measured camera motion in each frame, and determininga camera view direction adjustment angle based on the determined user'sview direction in each frame; and compensating for the camera viewdirection by using the determined camera view direction adjustment anglein each frame.
 38. The video stabilization method of claim 36, whereinthe stabilizing of the video of the camera comprises: measuring a cameramotion in each frame of the frame sequence of the video; determining atarget area of a view direction in the frame sequence; generating acamera motion path of the frame sequence by using the measured cameramotion in each frame and determining a camera view direction adjustmentangle based on the view direction corresponding to the determined targetarea by using the generated camera motion path; and compensating for thecamera view direction by using the determined camera view directionadjustment angle in each frame.
 39. The video stabilization method ofclaim 36, wherein the stabilizing of the video of the camera comprises:measuring an inter-frame camera motion based on a difference angle of arelative camera view direction with respect to a reference camera viewdirection in each frame of the frame sequence of the video; determininga target area of a view direction in the frame sequence; generating acamera motion path of the frame sequence by using the measuredinter-frame camera motion and determining a camera view directionadjustment angle based on the view direction corresponding to thedetermined target area by using the generated camera motion path; andcompensating for the camera view direction by using the determinedcamera view direction adjustment angle in each frame.
 40. The videostabilization method of claim 36, wherein the receiving of the selectionrequest from the user comprises determining a stabilization level basedon the margin area size indicated by the user.
 41. The videostabilization method of claim 36, wherein the receiving of the selectionrequest from the user comprises determining a margin area size based onthe stabilization level indicated by the user.
 42. The videostabilization method of claim 36, wherein the receiving of the selectionrequest from the user comprises receiving a selection request for apredetermined combination of the margin area size and the stabilizationlevel from the user.
 43. The video stabilization method of claim 36,wherein the stabilizing of the video of the camera comprises limiting atleast one of a magnitude of the measured camera motion, a magnitude ofthe determined camera view direction adjustment angle, and a variationaccording to the compensation for the camera view direction based on theat least one of the margin area size and the stabilization level.
 44. Avideo stabilization method comprising: stabilizing a video of a cameraby measuring a relative inter-frame camera motion based on a referencecamera view direction in a frame sequence of the video, determining acamera view direction adjustment angle based on a view directioncorresponding to a target area, and compensating for a camera viewdirection by using the determined camera view direction adjustment anglefor each frame in the frame sequence; measuring a stabilization-relatedindex while performing the stabilizing of the video of the camera; andwarning in real-time if the stabilization-related index exceeds apredetermined threshold.
 45. The video stabilization method of claim 44,wherein the stabilization-related index comprises at least one of amagnitude of a margin size generated by a video stabilization operation,a magnitude of a video camera motion sensed by a sensor, and a magnitudeof the measured inter-frame camera motion.
 46. The video stabilizationmethod of claim 44, wherein the measuring of the stabilization-relatedindex comprises measuring the stabilization-related index during atleast one of the inter-frame camera motion measuring operation, thecamera view direction adjustment angle determining operation, and thecamera view direction compensating operation.
 47. A video stabilizationapparatus comprising: a camera motion measurer which measures aninter-frame camera motion based on a difference angle of a relativecamera view direction with respect to a reference camera view directionin each frame of a frame sequence of a video; a camera parameterdeterminer which generates a camera motion path of the frame sequence byusing the measured inter-frame camera motion and determines a cameraparameter for each frame by using the generated camera motion path; anda camera motion compensator which compensates for a camera motion byusing the determined camera parameter for each frame.
 48. A videostabilization apparatus comprising: a camera motion measurer whichmeasures a camera motion in each frame of a frame sequence of a video; acamera view direction adjustment angle determiner which generates acamera motion path of the frame sequence by using the measured cameramotion in each frame and determines a camera view direction adjustmentangle based on a user's view direction by using the generated cameramotion path; and a camera view direction compensator which compensatesfor a camera view direction by using the determined camera viewdirection adjustment angle for each frame.
 49. A video stabilizationapparatus comprising: a camera motion measurer for measuring a cameramotion in each frame of a frame sequence of a video; a view directiontarget area determiner which determines a target area of a viewdirection in the frame sequence; a camera view direction adjustmentangle determiner which generates a camera motion path of the framesequence by using the measured camera motion in each frame anddetermines a camera view direction adjustment angle based on the viewdirection corresponding to the determined target area by using thegenerated camera motion path; and a camera view direction adjustmentangle compensator which compensates for a camera view direction by usingthe determined camera view direction adjustment angle based on the viewdirection corresponding to the determined target area for each frame.50. A video stabilization apparatus comprising: a camera motion measurerwhich measures an inter-frame camera motion by determining a differenceangle of a relative camera view direction with respect to a referencecamera view direction in each frame of a frame sequence of a video; acamera view direction adjustment angle determiner which generates acamera motion path of the frame sequence by using the measuredinter-frame camera motion and determines a camera view directionadjustment angle based on a user's view direction by using the generatedcamera motion path; and a camera view direction compensator whichcompensates for the camera view direction by using the determined cameraview direction adjustment angle for each frame.
 51. The videostabilization apparatus of claim 50, further comprising a view directiontarget area determiner which determines a target area of a viewdirection in the frame sequence, wherein the camera view directionadjustment angle determiner generates the camera motion path of theframe sequence by using the measured inter-frame camera motion anddetermines a camera view direction adjustment angle based on the viewdirection corresponding to the determined target area by using thegenerated camera motion path, and wherein the camera view directioncompensator compensates for the camera view direction by using thedetermined camera view direction adjustment angle based on the viewdirection corresponding to the determined target area in each frame. 52.A video stabilization apparatus comprising: a view direction target areauser input unit which receives a selection request for a target area ofa view direction in a frame sequence from a user through a userinterface; a camera motion measurer which measures an inter-frame cameramotion based on a difference angle of a relative camera view directionwith respect to a reference camera view direction in each frame of aframe sequence of a video; a camera view direction adjustment angledeterminer which generates a camera motion path of the frame sequence byusing the measured inter-frame camera motion and determines a cameraview direction adjustment angle based on the view directioncorresponding to the target area by using the generated camera motionpath; and a camera view direction compensator which compensates for thecamera view direction by using the determined camera view directionadjustment angle for each frame.
 53. A video stabilization apparatuscomprising: a margin area user input unit which receives a selectionrequest for at least one of a margin area size and a stabilization levelfor video stabilization from a user through a user interface; and avideo camera stabilizer which stabilizes a video of a camera bymeasuring a relative inter-frame camera motion based on a referencecamera view direction in a frame sequence of a video based on the atleast one of the margin area size and the stabilization level,determines a camera view direction adjustment angle based on a viewdirection corresponding to a target area, and compensates for a cameraview direction by using the determined camera view direction adjustmentangle for each frame of the frame sequence.
 54. A video stabilizationapparatus comprising: a video camera stabilizer which stabilizes acamera of a video by measuring a relative inter-frame camera motionbased on a reference camera view direction in a frame sequence of thevideo, determines a camera view direction adjustment angle based on aview direction corresponding to a target area, and compensates for acamera view direction by using the determined camera view directionadjustment angle for each frame in the frame sequence; astabilization-related index measurer which measures astabilization-related index while performing the stabilization of thecamera; and a video stabilization warning unit which warns in real-timeif the stabilization-related index exceeds a predetermined threshold.55. A non-transitory computer-readable recording medium storing acomputer-executable program for performing the video stabilizationmethod of claim
 1. 56. A non-transitory computer-readable recordingmedium storing a computer-executable program for performing the videostabilization method of claim
 11. 57. A non-transitory computer-readablerecording medium storing a computer executable program for performingthe video stabilization method of claim
 16. 58. A non-transitorycomputer-readable recording medium storing a computer-executable programfor performing the video stabilization method of claim
 21. 59. Anon-transitory computer-readable recording medium storing acomputer-executable program for performing the video stabilizationmethod of claim
 35. 60. A non-transitory computer-readable recordingmedium storing a computer-executable program for performing the videostabilization method of claim
 36. 61. A non-transitory computer-readablerecording medium storing a computer-executable program for performingthe video stabilization method of claim 44.